Detecting actions in untrimmed videos should not be limited to a small, closed set of classes. We present a simple, yet effective strategy for open-vocabulary temporal action detection utilizing pretrained image-text co-embeddings. Despite being trained on static images rather than videos, we show that image-text co-embeddings enable openvocabulary performance competitive with fully-supervised models. We show that the performance can be further improved by ensembling the image-text features with features encoding local motion, like optical flow based features, or other modalities, like audio. In addition, we propose a more reasonable open-vocabulary evaluation setting for the ActivityNet data set, where the category splits are based on similarity rather than random assignment.

In this paper, we propose an end-to-end Retrieval-Augmented Visual Language Model (REVEAL) that learns to encode world knowledge into a large-scale memory, and to retrieve from it to answer knowledge-intensive queries. REVEAL consists of four key components: the memory, the encoder, the retriever and the generator. The large-scale memory encodes various sources of multimodal world knowledge (e.g. image-text pairs, question answering pairs, knowledge graph triplets, etc) via a unified encoder. The retriever finds the most relevant knowledge entries in the memory, and the generator fuses the retrieved knowledge with the input query to produce the output. A key novelty in our approach is that the memory, encoder, retriever and generator are all pre-trained end-to-end on a massive amount of data. Furthermore, our approach can use a diverse set of multimodal knowledge sources, which is shown to result in significant gains. We show that REVEAL achieves state-of-the-art results on visual question answering and image captioning.

用于评估有条件自然语言生成的传统自动化指标使用单个生成的文本和最佳匹配的金标准地面真相文本之间的成对比较。当有多个基础真相可用时，分数将使用参考中的平均或最大操作进行汇总。尽管这种方法在地面真相数据中的多样性（即有条件文本的分布的分散）可以归因于噪声，例如自动语音识别中，但在地面上的多样性的情况下，它不允许进行强有力的评估。真理代表模型的信号。在这项工作中，我们认为现有的指标不适合诸如视觉描述或摘要之类的域，而地面真理在语义上是多样的，并且这些字幕中的多样性捕获了有关上下文的有用的其他信息。我们提出了一种新的范式，用于对条件语言生成模型的多键入评估以及一个新的指标家族，该指标家族使用每种少量样本集比较参考和模型生成的字幕集的分布。我们通过视觉描述中的案例研究证明了方法的实用性：我们在其中证明现有模型优化了单描述质量而不是多样性，并获得了对采样方法和温度影响如何描述质量和多样性的一些见解。

我们提出了IM2NERF，这是一个学习框架，该框架可以预测在野生中给出单个输入图像的连续神经对象表示，仅通过现成的识别方法进行分割输出而受到监督。构建神经辐射场的标准方法利用了多视图的一致性，需要对场景的许多校准视图，这一要求在野外学习大规模图像数据时无法满足。我们通过引入一个模型将输入图像编码到包含对象形状的代码，对象外观代码以及捕获对象图像的估计相机姿势的模型来迈出解决此缺点的一步。我们的模型条件在预测的对象表示上nerf，并使用卷渲染来从新视图中生成图像。我们将模型端到端训练大量输入图像。由于该模型仅配有单视图像，因此问题高度不足。因此，除了在合成的输入视图上使用重建损失外，我们还对新颖的视图使用辅助对手损失。此外，我们利用对象对称性和循环摄像头的姿势一致性。我们在Shapenet数据集上进行了广泛的定量和定性实验，并在开放图像数据集上进行了定性实验。我们表明，在所有情况下，IM2NERF都从野外的单视图像中实现了新视图合成的最新性能。

th骨海星（COTS）爆发是大屏障礁（GBR）珊瑚损失的主要原因，并且正在进行实质性的监视和控制计划，以将COTS人群管理至生态可持续的水平。在本文中，我们在边缘设备上介绍了基于水下的水下数据收集和策展系统，以进行COTS监视。特别是，我们利用了基于深度学习的对象检测技术的功能，并提出了一种资源有效的COTS检测器，该检测器在边缘设备上执行检测推断，以帮助海上专家在数据收集阶段进行COTS识别。初步结果表明，可以将改善计算效率的几种策略（例如，批处理处理，帧跳过，模型输入大小）组合在一起，以在Edge硬件上运行拟议的检测模型，资源消耗较低，信息损失较低。

横梁面部识别（CFR）旨在识别个体，其中比较面部图像源自不同的感测模式，例如红外与可见的。虽然CFR由于与模态差距相关的面部外观的显着变化，但CFR具有比经典的面部识别更具挑战性，但它在具有有限或挑战的照明的场景中，以及在呈现攻击的情况下，它是优越的。与卷积神经网络（CNNS）相关的人工智能最近的进展使CFR的显着性能提高了。由此激励，这项调查的贡献是三倍。我们提供CFR的概述，目标是通过首先正式化CFR然后呈现具体相关的应用来比较不同光谱中捕获的面部图像。其次，我们探索合适的谱带进行识别和讨论最近的CFR方法，重点放在神经网络上。特别是，我们提出了提取和比较异构特征以及数据集的重新访问技术。我们枚举不同光谱和相关算法的优势和局限性。最后，我们讨论了研究挑战和未来的研究线。

荆棘冠的海星（婴儿床）爆发是珊瑚损失的主要原因是巨大的障碍礁（GBR），并且正在进行大量监测和控制计划，以试图管理生态可持续水平的COTS群体。我们释放了GBR上的COTS爆发区域的大规模注释的水下图像数据集，以鼓励机器学习和AI驱动技术的研究，以改善珊瑚礁秤上的COTS群体的检测，监测和管理。该数据集发布并托管在一次竞争中，挑战国际机器学习界，并从这些水下图像中的COTS检测的任务挑战。

This paper introduces a video dataset of spatiotemporally localized Atomic Visual Actions (AVA). The AVA dataset densely annotates 80 atomic visual actions in 430 15-minute video clips, where actions are localized in space and time, resulting in 1.58M action labels with multiple labels per person occurring frequently. The key characteristics of our dataset are: (1) the definition of atomic visual actions, rather than composite actions; (2) precise spatio-temporal annotations with possibly multiple annotations for each person; (3) exhaustive annotation of these atomic actions over 15-minute video clips; (4) people temporally linked across consecutive segments; and (5) using movies to gather a varied set of action representations. This departs from existing datasets for spatio-temporal action recognition, which typically provide sparse annotations for composite actions in short video clips.AVA, with its realistic scene and action complexity, exposes the intrinsic difficulty of action recognition. To benchmark this, we present a novel approach for action localization that builds upon the current state-of-the-art methods, and demonstrates better performance on JHMDB and UCF101-24 categories. While setting a new state of the art on existing datasets, the overall results on AVA are low at 15.6% mAP, underscoring the need for developing new approaches for video understanding.

In this paper, we propose a novel technique, namely INVALIDATOR, to automatically assess the correctness of APR-generated patches via semantic and syntactic reasoning. INVALIDATOR reasons about program semantic via program invariants while it also captures program syntax via language semantic learned from large code corpus using the pre-trained language model. Given a buggy program and the developer-patched program, INVALIDATOR infers likely invariants on both programs. Then, INVALIDATOR determines that a APR-generated patch overfits if: (1) it violates correct specifications or (2) maintains errors behaviors of the original buggy program. In case our approach fails to determine an overfitting patch based on invariants, INVALIDATOR utilizes a trained model from labeled patches to assess patch correctness based on program syntax. The benefit of INVALIDATOR is three-fold. First, INVALIDATOR is able to leverage both semantic and syntactic reasoning to enhance its discriminant capability. Second, INVALIDATOR does not require new test cases to be generated but instead only relies on the current test suite and uses invariant inference to generalize the behaviors of a program. Third, INVALIDATOR is fully automated. We have conducted our experiments on a dataset of 885 patches generated on real-world programs in Defects4J. Experiment results show that INVALIDATOR correctly classified 79% overfitting patches, accounting for 23% more overfitting patches being detected by the best baseline. INVALIDATOR also substantially outperforms the best baselines by 14% and 19% in terms of Accuracy and F-Measure, respectively.

The recent increase in public and academic interest in preserving biodiversity has led to the growth of the field of conservation technology. This field involves designing and constructing tools that utilize technology to aid in the conservation of wildlife. In this article, we will use case studies to demonstrate the importance of designing conservation tools with human-wildlife interaction in mind and provide a framework for creating successful tools. These case studies include a range of complexities, from simple cat collars to machine learning and game theory methodologies. Our goal is to introduce and inform current and future researchers in the field of conservation technology and provide references for educating the next generation of conservation technologists. Conservation technology not only has the potential to benefit biodiversity but also has broader impacts on fields such as sustainability and environmental protection. By using innovative technologies to address conservation challenges, we can find more effective and efficient solutions to protect and preserve our planet's resources.